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INTRODUCTION 

It is becoming increasingly clear in the minds of 
those who face the practical problems of improving 
, instruction, that t ext books usually determine, the 
BiifMPiw nr failure of anv educational met hod. And 
yet the various problems involved in text book making 
have been attacked mainly through the hazardous 
and wasteful method of trial and error, or by the 
method of rough judgments. Recently, however, 
an inclination to regard the selection of text books 
as a matter for scientific study has become prevalent. 
Every~study dealing with students or social values or 
economy in learning has been promptly seized upon 
by authors or publishers either to guide the construc- 
tion of new books or to justify books already 
published. 

Studies of social utility in the various school sub- 
jects have been particularly effective in changing the 
content of text books. This is easily seen in the 
reconstruction of books in spelling, and in the reor- 
ganization of the texts in arithmetic. It is now 
practically impossible to sell spelling books which do 
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6 TEXTBOOK SELECTION 

not at least claim to be based upon the recent scientific 
studies of the vocabulary of correspondence. 

There is also rapid response to varying types of 
educational theory. Both the subject matter and 
the organization of books are "effected as the propa- 
ganda for new theories becomes wide spread. How- 
ever, both publishers and superintendents are coming 
more and more to distrust mere philosophizing as a 
method of determining the organization of school 
books; the publisher because these philosophical 
theories are often transitory; the superintendents 
because books based upon these theories are often 
found to be unsatisfactory. It is becoming' increas- 
ingly clear that all theories for the construction of 
text books must line up with the criteria now available 
from the scientific appraisals of social needs, and with 
the criteria available from the experimental studies in 
the economy of learning. 

These studies in the economy of learning, including 
the study of performance tests, are of the deepest 
significance. Some data are now available in almost 
every field and kind of learning. They range all the 
way from those learning the more elusive problems 
of determining tastes and interests to the somewhat 
simpler problems involved in determining practice 
exercises for specific performances such as hand 
writing. The two studies reported in this book are 
excellent examples of the utilization of scientific 
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TEXTBOOK SELECTION 7 

techniques in determining student reactions to text 
book materials. 

At a time when a new emphasis upon motivation 
is being made, Dr. Knight's study of students' choices 
in literature is very timely. We have only begun to 
appreciate the importance of utilizing the method of 
judgments of choices under properly controlled condi- 
tions. Certain aspects of learning are particularly in 
need of just such appraisals. It is fortunate to have 
an example which does not end merely as a statement 
of a general principle but in a helpful, concrete 
recommendation which can be immediately utilized 
by all teachers of high school English. 

It was inevitable from the first that the various 
studies of economy in silent reading would react upon 
the construction of text books. Dr. Franzen's 
investigation was made in the practical emergency of 
text book selection. It illustrates what may be 
expected more and more in the way of substituting 
data for opinion in the solution of the problems 
involved in text book adoptions. There is reason to 
believe that in the next few years we may expect 
much experimental activity along these lines. 

Both of the studies suggest the interesting possibil- 
ity of subjecting manuscripts to some such experi- 
mentation before printing. It would seem that authors 
owe this to the publishers, and both authors and 
publishers owe this to the public. Of course, scientific 
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8 TEXTBOOK SELECTION 

studies take time and are expensive. It is not unrea- 
sonable to expect that the profession of teaching is 
now on such a basis that the rewards for taking such 
pains will make the research worth while. 

Ernest Horn. 
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CHAPTER ONE 

PRINCIPLES OF TEXT SELECTION 

The major criteria for the selection of text-books 
are five : 

(a) The factor of interest. 

(6) The factor of comprehension. 

(c) The permanent methods ""bf study involved in 
the text. " — 

(d) The permanent value of the content, 
(c) The mechanic al construc tion of the text. 
These criteria possess determinative importance in 

the opinion of educators. There is no one way of 
expressing these criteria. Methods of statement differ 
but the meat of the matter is constant. We find 
the above statement convenient for our exposition 
but claim no particular rhetorical superiority for it. 
At present, it is difficult to apply these criteria in 
a particular situation. The application of general 
principles to concrete situations has always been 
troublesome. Educators find themselves in no uncom- 
mon impasse when they attempt to apply readily 
accepted principles to the specific problems of their 
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profession. The purpose of this monograph is a 
double one; first to state the fundamental propositions 
which must underlie sound practice of text-book 
construction; second to give two researches which are 
at least moderately successful attempts at such appli- 
cation. The researches are (a) the application of the 
broad doctrine of interest to the problem of selecting . 
texts for high school English Literature work, (b) 
the application of the factor of comprehension to the 
selection of geography texts for elementary schools. 

The history of public education in the United States 
is very far from a monotonous report of blunders and 
inadequacies. A study of text-book construction and 
of the methods of text-book selection even now preva- 
lent, however, gives the student of education reason 
to prophesy much increase in the genuine adequacy of 
public school text-books. Further we are sure that 
the selection of a particular text in preference to other 
available books is about to become far less a matter of 
guess and poorly based opinion and far more a matter 
of deliberate choice in the light of sound criteria of 
text-book excellence. 

Text-book selection is obviously a process of making 
choices, and choices can be made in many ways. It 
is quite possible that certain texts have been chosen 
not because they were better tools of instruction than 
other books, but because the salesman urging their 
adoption had a more persuasive vocabulary, more 
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agile and plastic sales methods, or was politically 
more canny than his competitors. Other texts have 
aad large sales because they were or professed to be 
exponents of some pedagogical doctrine which momen- 
tarily hypnotized the buying agent. Better than the 
above, many texts have been adopted by schools 
because the proper authority, having studied the 
matter deliberately, chose those particular books. 

Fundamental Propositions 

The thoughtful school administrator will pre- 
sumably subscribe to the following four propositions 
as fundamental to sound practice in text-book selection. 

I. Irrelevant differences pertaining to texts should 
be utterly neglected, viz. pressure from salesmen, 
exhilarating sales tactics. Texts should be chosen in 
light of the five determinative criteria of good text 
construction, namely: 

(a) The factor of interest. 

(6) The factor of comprehension. 

(e) The permanent methods of study involved in 
the text. 

(d) The permanent value of the content. 

(e) The mechanical construction of the text. 

■ II. Objective evidence should be given preference 
over subjective opinion wherever objective evidence 
can be reasonably obtained. 
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14 TEXTBOOK SELECTION 

III. Wherever Bubjective opinion ia resorted to the 
final judgment should not be the result of the wisdom 
of any one person. Final judgment should be the 
product of the pooled votes of several competent 
judges. Among these judges should be at least one 
class room teacher who is to use the book. The more 
mutually independent the judgments are the more 
stable the final result. 

IV. Wherever subject opinion has to be substituted 
for objective measurement, especial care should be 
taken to get the best opinion possible. A very good 
judge may sometimes give a very hasty opinion. 

Objective Evidence 

Proposition I, using relevant criteria, and proposition 
III, using the insight of more than one judge, need no 
further discussion. The main part of this book is an 
attempt to apply proposition II to actual, concrete 
school situations. Objective evidence is usually 
better than subjective opinion. Many of the decep- 
tions of opinion have been pointed out of late. But 
the objective evidence has its weak points also. Not 
all evidence is good evidence. Mere quantity of 
evidence does not involve in a mysterious way a 
guarantee that the evidence is reliable or consistent 
or determinative. Objectivity of evidence never 
compensates for the errors which come from wrong 
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methods of gathering the evidence or mistakes in the 
interpretation of the ever so objective data. 

It is hoped that the two problems of text-book 
selection dealt with in this book will be helpful not 
only because of the particular findings but also because 
of the exposition and discussion of the methods used 
to gain objective evidence concerning texts. 

The methods used are of general application. 
Those who are responsible for text-book selection* 
should know the reading difficulties not only of geog-l 
raphies, but also of all texts, especially arithmetic,' 
histories, and science books. 

One of the problems is the determination of the 
reading difficulty of geographies. Obviously a child , 
with fifth grade reading ability will work to dis- 
advantage if he uses a text which assumes a reading 
ability rarely found below the seventh grade. 

The fourth fundamental proposition of text-book 
selection is "Where subjective opinion has to be 
used, care must be taken to obtain the best opinion 
possible." This means the abandoning of general 
opinions and the persistent analysis and estimate 
of the specifics of each text judged. 

We no longer rate a school building as a whole. 
We judge it for ventilation, for play space, for class 
rooms, for many elements which taken all together 
make a school house. So with text-books, we should 
make our judgments not in general, but in particular. 
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It is apparent that score cards for text-books ire useful 
instruments to aid in making analyzeoSjjIdgments. 
Unfortunately such score cards are not in general use. 
In fact for many texts they are not in existence. 

The Use op Rating Score Cabds 

It is clear that the comparison of one text-book with 
another will be complete when both texts have been 
measured in the light of the five criteria. The whole; 
rating of text-books can finally be expressed in mathe- 
matical terms on a score card. For purposes of 
making a decision as to which of several texts to 
adopt the value of each text can, theoretically, be 
expressed in one figure. The figure would be the sum 
of the scores given to each of the five criteria if each 
criterion were considered as important as every other 
criterion. Weighted scores would be needed if the 
criteria were of unequal value. That these five 
criteria are of unequal value seems to be the case. 
From a pool of many judgments we assign following 
point values: 

Interest 200 

Comprehension 250 

Permanent value of subject matter . 250 

Value of method . . . 200 

Mechanical elements 100 

Total 1000 
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Score cards or rating schemes of one kind or another 
are generally used to get in orderly fashion expert 
opinion. 1 Many decisions in practical school affairs 
will have to be based on opinion rather than other 
data for some time to come. Wherever this is the 
case it becomes exceedingly important to obtain the 
best opinion in the best way possible. By best 
opinion we mean the judgment of those not only who 
know most about the matter but the opinion of 
enlightened judges who take the trouble to make the 
best judgment they can make. Here is where the 
score card is of value. Opinion expressed through 
score card ratings will not give insight the judge 
himself does not possess, but opinion expressed 
through score card ratings has a better chance of 
being opinion up to the limit of insight than unanalysed 
opinion has. Thus we see that the worth of opinion 
varies in merit not only with the true wisdom of the 
judge but, also, is a function of the method of obtain- 
ing the opinion. Analysed opinion through score 
card methods increases the chances for a judge to get 
all his goods to market. The better the score card 
the better the judgment will be. A Biology Score 
Card, one of a series, is given here. Its use makes 
judgment better because it forces the judge to con- 

* Opinion of any degree of expertness should be resorted to 
only^in the absence of evidential data. 



aii^cd by Google 



18 TEXTBOOK SELECTION 

aider many aspects of his problem he might otherwise 
neglect. An even better score card would make for 
even better judgments. 
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Tentative Score Cabs fob Judging Value or Biology 
Tkxtb {Continued) 



B. Difficulty of concept* 

1. Degree of novelty 

3. Degree of involved presentation . 

C. Difficulty of graphs and illustrative 

materia) . • 

III. Permanent Value of Subject Matter... 

(If there ia experimental evidence 
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3. Stimulation to further work — 
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Tentatto Scokjo Card fob Judging Valvb or Biology 
Tmcto (Continued) 



a. Sufficient data to make induc- 

tion possible 

b. Bummari™ 

d. Illustrations and maps. , .. 

2. Opportunity for application of 
■eneral principles 

a. Suggestions far outside wo 

b. Claaoroo in exercises . . 

B. Correlation with other subject 

ter 

1. Stimulus to more reading. ,. . 

2. Relation to other work 

V. Mechanical elements 

(Use objective measurements and norma 
to }ndf e these valu«> wherever possi- 
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A. Siae »nd clearness of print of test. . . 

B. Distinctness of pictures 

C. Sise and clearness of print of foot 

D. Site and clearness of print of mar- 

ginal notes and indices 

B. Width of margins 

F. Length of lines 

I. rHie and shape of book 
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APPLYING THE CRITERION OF INTEREST TO 

HIGH SCHOOL ENGLISH LITERATURE 

TEXTS 

Section Onb 

The business of this study is to do two things: 

A. First. — To name and classify in order of inter- 
est the best liked books commonly used in high 
school literature work. This list will be referred to 
as the Preferred List. 

Name in order of interest the next best liked books 
commonly used. This list is referred to as the Second 
Preferred List. 

Name in order of interest the third best liked 
books. This list will be called the Substitute List. 

Name in order of violent dislike books commonly 
used in high schools. This list will be referred to as 
the Free Reading List. 

B. The second section of our business is to (1) Defend 
the proposition that the Preferred List of books should 
form the basis or core of the English literature work 
in the high school, that books in the Second Pre- 
ferred List should be used next frequently, that the 
Substitute List may be drawn from after the Pre- 
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ferred and Second Preferred are exhausted and that 
books in the Free Reading List should never be used. 1 

This may be restated as follows: The best available 
criterion to use in selecting English literature books 
is the amount of interest students have in the books, 
considering only those books which are now in 
common use. 

(2) Present and interpret data calculated to establish 
the validity of our lists. 

The Prefebued List 

In the second section of this study sufficiently valid 
data are presented to warrant the assertion that the 
following are the best liked books commonly used in 
high schools of Iowa;*' ' w 

Careful students who wish to inquire into the factual 
basis of our lists before reading the lists will do better 
if they read the second section first. The busy 
1 Books in the Free Reading List should never be used as a 
class text. These books should be in the library for the use of 
those very few students who wish to read them. 

* Our data, we hasten to point out, are based on generous 
samplings from high schools of Iowa. Our criterion of interest 
would apply strictly only to Iowa. The lists of books are gener- 
ally applicable in so far as any large group of students have 
about the same interests as large groups of Iowa students. 
Some exceptions in interest are probably important. For 
instance Uncle Tom's Cabin is highly interesting to Iowa high 
school pupils. Its interest rating would probably be much 
lower in Georgia or Louisiana. 
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school administrator can assume the validity of the 

evidence and content himself with a perusal of the 
lists. 

Table A.— Pbeferred List 

Reliability 

Interest of 

Nauh or Book Rating 1 Rating* 

Laddie 97 1.4 

Vicar of Wakefield 92 1.7 

Kenilworth 79 3.7 

Treasure Island 66 2.5 

Hamlet 63 2.5 

Comus 61 3.5 

Uncle Tom's Cabin 61 3.0 

Wild Animals I Have Known 60 5.1 

Ben Hut 59 3.0 

Last Days of Pompeii 59 4.1 

The Spy 59 3.6 

Romona 58 4.4 

King Lear 57 2.2 

The Crossing 55 4.3 

Snowbound 54 2.5 

Luck of Roaring Camp 54 4.0 

Macbeth 53 2.2 

Ivanhoe 53 3.8 

Carlyle's Essay on Burns 53 3.3 

1 Interest rating is in terms of the percentage of students, 
who, having read the book, declare that it was of distinct 
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Second Pkeferbbd List 

Incidents of a French Camp 53 4.2 

The Virginian 52 3.8 

Les Miserables 52 4.5 

The Crisis 51 3.1 

The Last of the Mohicans 51 2.6 

Up From Slavery 51 5.5 

Tom Sawyer 50 2.3 

Tale of Two Cities 50 3.0 

Silas Marner 49 2.2 

Man Without a Country 49 2.4 

Huckleberry Finn 49 2.7 

Washington's Farewell Address 49 3.3 

Ancient Mariner 48 2.2 

The Gold Bug 48 2.5 

Richard Carvel 48 5.3 

Miles Standish 47 3.2 

Robinson Crusoe 46 2.6 

Lorna Doone 46 4.0 

interest to them. Thus the higher the rating, the better the 
book according to our criterion. 

' Reliability of Rating ia to be read as follows: If many 
more estimates of the book were taken, the interest rating might 
go up or down as much as the reliability figure. The chances 
of its going up or down by an amount greater than the reported 
unreliability are very few. Thus The Vicar of Wakefield has an 
interest rating of 92 and an unreliability of 1.7, which means 
that aa much more data are obtained, the 92 may go up to 93.7 
or down to 90.3. The technique of ascertaining the reliability 
is fully dealt with in section two. 
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Free Reading List 

In Order of Demerit 

Contains books Never to be Used in High School 

Paradise Lost , 6 1.4 

Lay of the Last Minstrel 10 2.5 

Heroes and Hero Worship 11 2.8 

Easay on Warren Hastings 12 3.4 

Macauley's Essay on Johnson 12 2.6 

Autocrat of the Breakfast Table 13 2.0 

Vision of Sir Launfal 13 1.2 

Childe Harold 18 3.5 

Webster's Bunker Hill Speech 19 2.8 

Fairie Queene 20 2.9 

Essays of Elia 21 4.5 

Lays of Ancient Rome 21 4.4 

Utopia 23 4.5 

Thoreau's Walden 23 4.4 

Gareth and Lynette 23 2.9 

The Princess 24 3.5 

Tales of a Wayside Inn 24 2.7 

House of Seven Gables 24 2.1 

Pilgrim's Progress 25 2.5 



aii^cd by Google 



TEXTBOOK SELECTION 27 

Substitute List 

Contains in order of probable interest to students 
books less interesting than the First and Second Pre- 
ferred List, but of measurably greater interest than the 
books of the Free Reading List. The books are listed 
in order of interest merit. 

Twelfth Night 46 4.9 

Launcelot and Elaine 45 2.6 

Hoosier Schoolmaster 45 3.5 

Christmas Carol 44 2.5 

Hans Brinker 44 3.5 

Legend of Sleepy Hollow 43 2.1 

David Copperfield 43 2.7 

Joan of Arc 43 3.9 

Enoch Arden 43 2.8 

Lincoln's Gettysburg Address 42 2.2 

Leatherstocking Tales 42 3.6 

Merchant of Venice 41 2.0 

Burke's Conciliation 41 2.9 

Midsummer-Night's Dream 40 3.6 

Prisoner of Chillon 40 3.1 
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The interest ratings of 27 other books frequently- 
read in high schools have been determined. The 
First, Second Preferred and Substitute Lists contain 
those of highest ratings. These lists contain more 
books than would ordinarily be used in a four-year 
high school. It seemed needless to print the others. 
These interest ratings may be obtained on request. 

Notes on the Lists 

The criterion used in setting up an order of merit 
of English literature texts is pupils' interest in the 
text. This proposition is defended in Section II. 
Any one whose choice of texts in the practical situa- 
tion of actual use in high schools is influenced by 
these lists should bear the following in mind : 

(a) The interest which students have in the texts 
is not the only criterion to be used in deciding which 
one of two possible books to use. Other things being 
at all equal, however, it is better psychology and 
sounder common sense to prefer interesting material 
over material of no interest or positive dislike. 

(&) Not every book that could be used in high 
school is included in this study. No book is dealt with 
here unless it has been reported by at least eighty 
of the high school graduates who rated the list. Thus 
we are concerned only with books which are already 
fairly commonly accepted as worth while. This 
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matter could be stated : English literature pro- ■ 
feasors and teachers have on the whole approved of 
many books for high school work. More books are 
generally approved than can possibly be read by any 
one student. Of books accepted as useful which shall 
be chosen? Those of greatest interest to the pupils 
is our thesis. 

(c) It is not certain that the lists are equally sound 
for all sections of the country. In some instances we 
know radical variations in interest would be very 
probable. Children's interests must vary with the 
traditions and interests of the'section in which they 
live; how much is not known. Probably books 
vary in their relative universality of appeal. Thus 
Shakespeare may be of about the same interest 
strength everywhere, whereas Uncle Tom't Cabin may 
have high interest in some sections and practically 
none in others. We are well within the facts on the 
other hand to suspect the absence of radical and sharp 
changes in interest. The children of one county 
have about the same interests as those in a contiguous 
county. There is no apparent reason that on a basis 
of children's interests there should be endless variation 
within the same state. It might be added that there 
is probably less difference in interests in city versus 
rural youths than is ordinarily assumed. To differ- 
entiate books on a basis of sex is idle, for boys and 
girls have to read the same books in most high schools. 
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There is do reason why our lists might not very closely 
approximate at least mid-western lists. 

(d) Other criteria should be used in making up a 
list for a particular high school. It is sound to hold 
that other things being equal books in the Preferred 
List should be read sometime during high school. The 
books in excess of this Preferred List should be chosen 
from the Second Preferred List. However, a book 
with an interest rating of 54 % poorly taught will prob- 
ably do less service than a book with an interest rating 
of 48% well taught. Thus, while no book should be 
used from the Free Reading List, in a particular situa- 
tion should a teacher much prefer a 48% book to a 
54% book it is good sense to let her teach it, for the 
differential of the teacher's attitude is quite probably 
more important than a small difference in percentage 
of interest. 

(e) Should the preponent elements in the situation 
be a set of slightly lower interest books, on hand, and 
in good condition, a set of slightly higher interest books 
available only through purchase, a supply of funds 
so limited that the purchase of the new books means 
going without some other good thing; then to delay 
for a time the desired replacement is probably entirely 
defensible. 

(J) The interest which one has in any book is due 
not only to the character of the book but also to the 
character of the reader and the method of reading. 
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Thue the same book though read under equivalent 
conditions will not have the same amount of interest 
for all students, not because the book varies but 
because students differ greatly in what they want out 
of a book. Differences may lie in many fields such 
as home training, type of play indulged in, amount of 
travel, ability to read, type of fantasies or day dreams 
enjoyed, etc. ^ Thus, even the most interesting books 
on the list can not be expected to interest all pupils. 
We know they will not. We are pretty sure the books 
in the Preferred List wil> very likely be actually dis- 
liked by some students. In addition a book which is 
generally preferred to a second one may in some 
peculiar instances be less liked by a class or school 
than the second. However the theory of proba- 
bility as used in this study is a sound technique in 
curriculum construction. 

Exceptions need not greatly disturb us. Obviously 
a book tin and of itself may be inherently more 
interesting to pupils than some other. But if the 
other book happens to be well taught and the book 
inherently better happens to be grossly mistaught, 
then the inferior book may very likely appear to be 
better. 

(g) A certain variety in texts is undoubtedly 
desirable. Had the Preferred Lost on a basis of inter- 
est been all drama, or all poetry, or all heroic novel, 
practical consideration would force us to draw heavily 
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od other school literature. But the Preferred List as a 
matter of fact is pretty well balanced as far as kinds 
of literary content are concerned. And many wide 
deviations from this list can be comfortably accounted 
for by the consideration that while what large groups 
tend to like, other large groups do not tend to dislike, 
in individual cases no one would expect all pupils to 
like the same books and in the same relative order of 
interest. These lists do not pretend to be correct 
for all students, but as far as we now know they will 
strongly tend to be better than any other list. 

(ft) It is not useful to think of the most interesting 
book as the "best" book or the least interesting book 
as the "worst" book. The most interesting book is 
Laddie. No one would claim that Laddie is the best 
book in the sense that "if only one book was to be 
read in high school this book should be read." Such 
choices and such "ifs" do not have to be made. 
Many books are read. The question is which group is 
best. In the best group Laddie finds a place and 
Paradise Lost does not. This placing is, of course, 
according to our criterion of interest. 
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Reapers' Interebt the Criterion 1 

The criterion used, in the construction of these lists 
is the interest of the student as reported. 

No one aware of the administrative exigencies of a 
high school would contend that "what pupils like" 
should be the sole criterion for admitting a book, a 
subject, a technique, or a rule into the total economy 
of high school procedure. Neither would one suspect 
that what pupils dislike should be the sole criterion 
for dropping a book, a subject, a technique, or a rule 
from the school. 

It is hardly conceivable that any teacher is so 
unenlightened as to hold that disagreeable tasks are 
worth while because of their disagreeableness. A 
task may be distasteful and yet worth while, but its 
worth-while-ness is certainly due to other factors than 
its unpleasing nature. 

Theoretically, most competent English teachers 
would agree that one of the criteria safely used to 
select texts would be students' interest, but there 
would be little agreement on the amount of influence 
interest should have. If the books available should 

1 Theoretical consideration of the importance of interest is 
assumed. Personally, we view those who do not appreciate 
the pragmatic importance of interest in education as in need of 
instruction in the simple principles of the teaching process. 

Nor are we concerned here with the objectives of our English 
Literature work. 
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be scored on a basis of 100 points, would literary- 
merit be assigned 60 points, moral teaching 25, and 
interest 25, or would it be reading difficulty of the 
composition 50, literary merit 15, interest 15, amount 
book is used in other schools 20, — or what relative 
importance would be assigned to interest? 

In our present ignorance of the details of scientific 
management in education, one would utterly despair 
of getting definite answers to questions of this kind. 
However answers to such questions must be obtained 
before we can speak of education as a science in the 
sense that civil engineering is a science. All would 
agree that interest is only one of several factors which 
properly influence selection. How influential a factor 
it should be has not yet been scientifically established. 

Our data show that the selection of English literature 
in the State of Iowa practically refuses to recognize the 
factor of interest at all. The frequency with which 
books are read in Iowa high schools and the frequency 
with which students report these books as interesting 
to them have practically no correlation other than 
that of chance. This roughly may be taken to mean 
that the more frequently read books are rated as less 
than average interest just as often as they are rated 
as above average interest, also that the less frequently 
read books receive an interest rating greater than the 
average as often as "less than the average" rating. 
Such a state of affairs would be justifiable if it could 
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be clearly shown that other criteria more important 
than students' interest determined the frequency with 
which books are read. That such criteria do not 
operate will be shown and evidence given later in this 
study. 

Assuming that the 783 reports on which this study 
is based are a fair sampling, the following examples of 
the strained relations if not absolute divorce between 
"how often books are read" and "how much books 
are liked " will be illuminating. 

The 19 books read with the greatest interest were 
recorded on the average 280 times. 

The 20 books read with enough interest to warrant 
being in a Second Preferred or First Substitute Lists 
were read on the average 247 times. 

The 20 books read with so little interest that they 
should not be read at all were read on the average 
203 times. 

The average interest rating of the Preferred List 
is 62.7%. 

The average interest rating of the Free reading List 
is 16.3%. 

Thus, while in frequency the shrinkage in use 
between the best and worst lists is only 17.6%, the 
shrinkage in interest is 46.4 %! The shrinkage in 
interest is approximately 2.6 times as great as the 
shrinkage in frequency of use between the Preferred 
and the Free Reading List. I doubt if any one would 
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want to defend this relationship. It certainly would 
never have happened if the choosers of text-books 
had known the interest in books before making their 



To those who may suspect that a study of only 
the best books and the worst in point of interest hides 
a more favorable state of affairs because it does not 
consider the middle books at all, the relation between 
frequency of use and amount of interest of all books 
will be given. 

The technique here is that of correlation. 

We digress to explain what is meant by correlation. 
Persons familiar with statistical theory should skip the 
next page. 

The coefficient of correlation is a measure of con- 
comitant variation between two series of facts about 
the same data. The limits of this coefficient are 
— 1 and +1- If the relationship is + lit means that 
there is a perfect matching. Thus if there were a + 1 
correlation between the frequency with which the 
books we are studying are read and the amount of 
interests pupils have in them, then the following 
conditions would hold: 

The book read most frequently would also be the 
most interesting; the book read least frequently would 
also be the least interesting; the book read with the 
fifth greatest frequency would also be the fifth book in 
interest; the book read next to the least often would 
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be the book next to the bottom in interest, and 
ao on. 

If there were a —1 correlation between frequency 
and interest then just the reverse would be true, i.e., 
the most read book would be the least interesting; the 
book fifth from the top in frequency of the reading 
would be fifth from the bottom in its interest rating. 

If the correlation were zero then there would be 
no relationship but a chance one between frequency 
and interest. Some much read books would be high 
in interest, others low, others average. Some books 
low in frequency would be low in interest, others high, 
others average in interest. Thus the sign and size of 
the correlation coefficient measures the amount of 
mutual relationship between the frequency with which 
books in our list are read and the amount of interest 
pupils have in these same books. 1 

Using only the books which were reported eighty 
times or more (the unreliability of ratings based on less 
than eighty gets too large for conservative study) we 
find the correlation between frequency of use and 
interest to be — ^04, practically zero, 

1 l\irtlier explanation of the correlation coefficient may be 
found in 

Thorndike. Mental and Social Measurements. 

Brown and Thomson. BnentiaU of Mental Measurement. 

Rugg. Statistical Method Applied to Education. 

Refer to index in each book. 
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This means that there is no tendency to read the 
more interesting books of toner than the non-interest- 
ing one. There is also no tendency to read the non- 
interesting ones any more frequently than the 
interesting ones. 

In view of the fact that the data give also a rating of 
positive dislike it is instructive to study the correlation. 
between the frequency with which books are read 
and the positive dislike for them. It should be 
pointed out that the positive dislike ratings are 
entirely separate from any interest ratings. They 
were made independently. The correlation is — .52 
between frequency of reading and positive dislike. 
This means that there ts a pretty definite factor of 
negative selection going on at present. The nearest 
approach to anything like scientific control that 
the data reveal is this fact that on the whole books 
which are positively disliked tend to be read less often 
and there is a slight tendency to read them less often 
in the order of the strenuouBness of dislike . 

Certainly the factor of interest should operate in the 
selection of texts. But it does not. We are not 
prepared to say that it is theoretically impossible to do 
good work in a subject in which the learner has no 
interest, but interest is a factor which facilitates 
learning. On the whole, interest in the work runs 
along with skill in it. With material such as litera- 
ture the interest factors are probably more important 
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than in more mechanical subjects where physical 
skill based on habit predominates. If reading books 
carries no interest but often even positive annoyance, 
there is little if any likelihood that reading goes on 
except under compulsion. The frequent task of 
reading inherently uninteresting material under school 
compulsion may account for whatever truth there 
is in the general feeling that adults on the whole 
read little literature of great merit. Very few high 
school graduates ever read the JSneid over again for 
the fun of it. We doubt if one high school graduate in 
fifty, who, having read Sesame and Lilies or Paradise 
Lost in high school, ever reads these books again, or 
reads other works by the same authors unless they are 
in turn teaching them in high school for a wage, to 
pupils who will never look at them or think of them 
again unless later as high school teachers, they in 
turn — and so on. L 

Several cross sections of the data are presented here 
to further illustrate the divorce between interest and 
frequency of reading certain books. 

Table B contains the 29 books read the most fre- 
quently. On these we have 350 reports or more. 

1 The way a book is taught must determine in part the inter- 
est pupils have in it. ■ This relation between how a book is 
taught and resulting interest is as yet unknown. 
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No. ot 

Turns Place on Interkbt 
Reportbd Name or Book Rating 1 

720 Vision of Sir Launfal Free Reading List 

607 Merchant of Venice Substitute 

552 Legend of Sleepy Hollow. . . . Substitute 

535 Lady of the Lake. No place 

529 Silas Marner Second Preferred 

523 Macbeth First Preferred 

523 Lincoln's Gettysburg Speech Substitute 

503 Ancient Mariner Second Preferred 

492 Ivanhoe First Preferred 

481 Courtship of Miles Standish. Second Preferred 

465 Great Stone Face No place 

460 The Raven No place 

459 Julius Caesar No place 

424 Man Without a Country Second Preferred 

414 Gold Bug Second Preferred 

411 House of Seven Gables Free Reading List 

397 Christmas Carol Substitute 

385 Last of the Mohicans Second Preferred 

383 Canterbury Tales No place 

383 Snowbound First Preferred 

380 Passing of Arthur No place 

378 Sketch Book No place 

375 Hamlet First Preferred 

369 Treasure Island First Preferred 

368 Thanatopsts No place 

368 Idylls of the King No place 

366 Robinson Crusoe Second Preferred 

366 Gray's Elegy No place 

357 Launcelot and Elaine Substitute 

1 "No place" means that the book has an interest rating 
better than the Free Reading List but not high enough to reach 
even our Substitute List. 
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Of the 29 most frequently read books (350 reports 
or more out of 783 students) 

5 are in our First Preferred List 
7 are in our Second Preferred List 
5 are in our Substitute List 
10 are in "Not Placed" list 
2 are in the Free Reading List 
Total 29 

Of the ten books'Jwhich had an interest rating 
of 45, one was read 640 times and one but 160 times. 
The other eight were rather evenly distributed 
between these extremes. Of the twelve books read 
with an equal frequency (280 times) one had an inter- 
est rating of 10, another of 95. And here the other 
ten were scattered between these extremes. 

All the data show how books with equal interest 
are read with highly variable frequency. Books read 
with equal frequency have highly variable interest 
value. 

So far we have shown that interest as such exerts 
no measurable influence on frequency of reading over 
the state of Iowa as a whole. We wish to present 
data to show that the operation of other definite 
criteria is not evident. 

If clean cut criteria were operating under conscious 
direction some one would know about it. Who does? 
What are the criteria? Reading difficulty is a very 
good criterion, but the reading difficulty of books 
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used in high school English literature has never been 
determined. Some one could very well determine this. 
Perhaps information value is thought by some to 
be an effective criterion which we have carelessly over- 
looked. We have yet to see a study of the information 
value of the books under consideration. Such a 
study might well be made. At present, however, this 
criterion is still a "poor relative" of opinion and a 
very much be-fussed relative into the bargain. It 
would be difficult to show that the much read books 
are higher in information value than the much liked 
books. Comparing the First Preferred List with the 
Free Reading List one might perhaps be impressed 
with the realism of the Preferred List in contrast to 
the fantasy of the Free Reading List. Possibly 
people have to be older than of high school age before 
they become sufficiently disillusioned about the world 
to find pleasure in fantasy. The high school pupil 
may be still satisfied with the world as it is, or nearly 
is. He may not yet be interested in Utopias, Para- 
dises Lost or Found, Fairie Queens or Visions, or he 
may be too old to be interested in them. 

UTEBABY MERIT NO CRITERION 

It is always possible to fall back on the well known 
defense of literary merit. It could be asserted, if 
one is careless enough about the facts, that the 
criterion of literary merit decides the frequency with 
which books on the whole are read in high school. 
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In one sense this is true. Probably no book is much 
read until by a vague consensus of opinion it is worth 
reading. But this does not effect our data because 
we have considered only books rather frequently 
read. They all possess enough literary merit to be 
generally used. But differences in literary merit in 
excess of a minimum amount of it to allow entrance 
of books into common use can not account for the 
frequencies. 

We have accepted literary merit in this sense, as 
a working criterion of a minimum amount. It is 
important to realize that in no other sense can we hold 
it to be influential. It ia not at present a criterion for 
two reasons. First, there is no objective measure to 
use as a criterion. There may be some day a scale for 
rating for literary merit. Up to now contributions on 
the matter have been much cluttered with opinions 
and not disciplined with scientific or orderly study. 
In short, every one has a literary rating all his own, 
(which rating we suspect is largely a rationalization of 
likes and dislikes by the individual doing the rating). 

Second. — If differences in literary merit decided 
differences of frequencies of use so that the most meri- 
torious book was read the most frequently, the second 
most meritorious book was read the next most fre- 
quently, and so on, then there would have to be a 
general agreement as to the differences in literary merit 
of the books concerned. There is not even an approxi- 
mation of agreement, as the following data ihow : 
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We have had five groups of judges rate the nineteen 
books composing the Preferred List and the nineteen 
books composing the Free Reading List for the quality, 
literary merit. The judges were in all instances igno- 
rant of the interest value of these books. They were 
urged to give their own opinion independent (as far as 
possible) of fame of the author, popular appeal of the 
book, etc. 

The value A was to be given to those books which 
were of preeminent literary value, the great books 
of the ages. 

The value E was to be given to those books which 
were of such poor quality that probably they should 
not be considered for serious work at all. 

The value C was to be given to books about half way 
between A and E. 

The value B was to be assigned to books better 
than C but of less merit than A. 

The value D was to be assigned to books worse than 
C but better than E. 

The additional instruction was given that in case 
of unfamiliarity with a book no rating need be given. 

The pooled results of these judgments of the literary 
value of the Preferred List (each book given a number) 
and of the Free Reading List (each book given a 
number) is presented. The reader is asked to dis- 
tinguish on a basis of literary merit which list of 
books is the best. 
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ListX 1 

Showing the distribution of judgments on nineteen 

books for literary merit. A, B, C, D, E, amounts of 

merit as defined in text. Numbers 1, 2, 3, etc. refer 

to books. 

ABODE T 

3 21 27 6 3 60 
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52 


46 
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11 
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10 
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33 
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17 


53 
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16 
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113 


18 


18 


51 


29 
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103 


19 


7 


17 


21 


12 


6 


63 



T 428 627 585 175 52 1817 

1 Whether this list is the Preferred or the Free Reading List 
will be stated later on. This postponement is to enable the 
reader to see the utter hopelessness of getting a "literary 
merit" when the books are given by number rather than name. 
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326 514 450 



aii^cd by Google 



TEXTBOOK SELECTION 47 

The reader will readily see that a great amount 
of sagacity would be necessary to determine whether 
last X or List Y is superior in literary merit. There is 
certainly not enough difference to outweigh the essen- 
tial difference injnterest value which exists between 
the two lists. 

The careful student will immediately wonder if our 
pooled ratings are not a mixture of inexpert and con- 
sequently valueless opinion. We shall therefore spread 
out the pith of each group of ratings to show that 
there is no difference of importance between the 
judgments of groups which vary in expertness. 

Group I ib a class In psychology of advertising. 
This group is not expert in matters of literary merit of 
English literature. Their judgments are: 

List X List Y 

of 369 judgments of 230 judgments 

on 19 texts on 19 texts 

21.4% judgments were A 12.2% were A 

35.6% judgments were B 38.7% were B 
31.5% judgments were C . 42.2% were C 

9.5% judgments were D 4.3% were D 

1.9% judgments were E 2.6% were E 

We interpret these data as showing that in the 
opinion of a psychology of advertising class List X 
is slightly superior to List Y in literary merit. 
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Group B is a class of college juniors preparing to 
teach in high schools. We assume that their judg- 
ments are slightly more expert than the judgments of 
Group A. 



List X 

of 095 judgments 
on 19 texts 



List Y 

of 803 judgments 
on 10 texts 



28.3% of judgments were A 19.9% were A 

36.8% of judgments were B 39.9% were B 

24.7% of judgments were C 29.0% were C 

6.6% of judgments were D 8.5% were D 

3.5% of judgments were E 3.6% were E 

In the judgment of Group B List X appears to be 
slightly better than List Y. The chief characteristic 
of this difference is however its negligible smallness. 
Other criteria than literary merit should obviously 
operate in deciding which list to uae. 

Group C is a graduate seminar in educational 
psychology. Most of the group are experienced 
teachers. They are presumably still more expert 
than either Group A or B. 
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List X List Y 

of 216 judgments of 222 judgments 

on 19 texts on 19 texts 

26.3% of judgments were A 36.5% were A 

35.6% of judgments were B 31.1% were B 

27.3% of judgments were C 23.9% were C 

9.3% of judgments were D 6.3% were D 

1.4% of judgments were E 2.3% were E 

Here List X has 61.9% of judgments B and A 
List Y has 67.6% of judgments B and A. 

There is a slight difference in total literary merit in 
favor of List Y. But the difference is so slight that no 
one could possibly judge it to be as important psycho- 
logically, pedagogically, or culturally from the teach- 
ing standpoint as is the difference in interest value 
determined by this study. 

Group D, our most expert group, is Doctor 
Craig's seminar in introduction to graduate work in 
English. Many of the group are instructors or have 
been instructors in English literature. They have 
profited by unquestioned expertnees in English 
literature through graduate study in English. 
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of 182 judgments of 176 judgments 

on 19 texts on 19 texts 

22.5% of judgments were A 14.2% were A 

21.9% of judgments were B 27.8% were B 

34.6% of judgments were C 38.6% were C 

15.9% of judgments were D 14.7% were D 

4.8% of judgments were E 4.5% were E 

Two facts are apparent: (1) the expert group rated 
both lists on the whole lower than the relatively less 
expert groups rated them; (2) in the judgment of the 
moat expert group List X and List Y are practically of 
equal literary merit. Perhaps List X is slightly 
better. In aU cases List X is our Preferred List and Y 
our Free Reading List. 

The force of the argument is now apparently 
adequate. 

When groups of judges varying in expertness are 
asked to pass on two lists of books the experts hand in 
about the same findings as the less expert. There is 
then no expert opinion as different from common 
knowledge. For if there were a useful expertness 
graduate students in literature would have it. They 
do not. Our blunt conclusion is: since judgments of 
literary merit on our Preferred List as a whole and on 
our Free Beading List as a whole can not be made to 
even suggest significant differences, then literary 
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merit as a criterion, except as we have already admitted 
it 1 is not determinative. This we take it leaves a clear 
field for the validity and usefulness of the lists pre- 
sented in the first of this study which are based on the 
criterion of interest. 1 

This lack of agreement in matters of merit is not 
surprising. After all literary merit or any of its 
synonyms contains but little useful truth. It is 
similar to such phrases as the Well-rounded Life, 
Education for Complete Living, True Culture, Good 
Citizenship, and the like. We assume that any phrase 
such as those contains a notion upon which many can 
agree as they can agree on the notion that two plus 
two equals four. No such notion is contained by such 
a phrase as Literary Merit, or Education for Complete 
Living, or Good Citizenship. These are verbal fog 
screens concealing chiefly differences of opinion. 
After all what may appear to you as a well-rounded life 

i Minimum amount to be admitted at all. 

1 If the reader should feel that there are a few books in the 
Free Reading List of such great merit that they should be 
read in spite of their low interest value he can satisfy himself 
as to the difference of opinion on the matter by attempting to 
pick out from the pooled judgments of List Y which books they 
are by the spread of judgments presented there. He might 
possibly pick number 6, 13, 17, which are Autocrat of the 
Breakfast Table, Utopia, Tale* of Wayside Inn. No one of 
these books are really liked by one pupil out of four who have 
read them. 
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may appear to your neighbor as the life of a prig, a 
selfish hoarder, a silly person, or what not. While his 
notion of a well-rounded life may seem to you to be a 
life too physical, too intellectual, too ethereal, or too 
worldly. And all the while we know you are both 
wrong and that our notion of the well-rounded life 
though sharply differing from both of yours is really 
correct. 

So with literary merit, perhaps that is meritorious 
which is liked or which satisfies, or in some other way 
leaves a characteristic impression on the reader. Aa 
you and your neighbor are forever satisfied with dif- 
ferent things, literary merit is one thing to you and 
another to him. 

Even though this explanation is faulty the cold 
fact remains that there is as yet too little agreement 
as to the varying worth of the books we are considering 
to use merit as a criterion. 

Third, — if this were at present a criterion, it cer- 
tainly is not effective. For who would say the follow- 
ing differences in frequency of reading were due to 
clear differences in literary value : 
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Vision of Sir 

Launfal read 720 times out of 783 reports 

King Lear read 185 times out of 783 reports 

Merchant of Venice. read 607 times out of 783 reports 

Hamlet read 375 times out of 783 reports 

Julius Caesar read 459 times out of 783 reports 

Treasure Island read 369 times out of 783 reports 

Sesame and Lilies. . .read 145 times out of 783 reports 
Gold Bug read 414 times out of 783 reports 

Another defense for the present situation would be 
that a balanced literary ration must be provided for 
high school pupils and the present practices are 
approximations of this balance. The necessity for 
balance is of course entirely proper and sound. But 
there is some difficulty in justifying the wide spread 
notions about just what a balanced ration is. How- 
ever, it is to be noted that there is at least variety in 
the Preferred List as well as in the other lists. There 
is a fairly" good variety both as to the type of literature, 
date of publications (i.e., age) and many of the books 
in our liBts have undisputed literary merit. We 
doubt if in the matter of balance the books most often 
read make any better presentation than do the books 
which when read are of distinct interest. 

The present situation in English literature as far 
as the frequency with which books are read ia con- 
cerned, is certainly not the result of conscious, 
purposeful, scientific control and direction, right or 
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wrong. It is a result of a confusion of influence hope- 
lessly interwoven. In addition it should be kept 
clearly in mind that even if other criteria were oper- 
ating still the present impotence of the interest factor 
can be viewed with philosophic calm only by the 
pedant. 1 

The evidence so far supports the following 
statements: 

The correlation between the criterion of interest 
and the frequency of reading is zero. 

The criterion of reading difficulty can not operate 
in a useful fashion for the difference in reading diffi- 
culty has never been determined. 

The criterion of useful information is not operative 
for the useful information rating of the books is too 
nebulous. 

Literary merit doubtless operates in the restricted 
sense of determining in some way the entrance of 
a book into use at all. But it does not determine the 
relative importance of books that are used, for there 

1 It can be shown statistically that the operation of definite 
criteria need not be- assumed to account for the present situa- 
tion. For the frequency with which books are read conform 
to the upper half of a pure chance curve within the probable 
error. The treatment of the data to prove this is deleted for we 
do not need it to prove our point concerning the absence of 
steering and the prevalence of drift at present. Moreover our 
interest is not to prove things are wrong but to sell better 
English literature choices to school managers. 
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is not sufficient agreement as to the measuring of 
literary merit. 

We repeat that the use of some criterion though 
imperfect is better than the vague and uncertain 
practices which now exist. If choices of high school 
literature texts were made in the light of the lists 
of this study the situation would not be made worse 
and would probably be genuinely improved. The 
criterion of interest is not only known but is of 
respectable importance in any teaching situation. 

Section II 

We have presented lists of literature texts. 

We have defended the use of the criterion of interest 
in Betting up lists. 

However, the lists are good only to the extent 
that the data upon which they are based are valid.' 
We now present a description of the original data. 

The freshmen English classes at the State University 
of Iowa were given as an exercise the filling out of the 
following blank. 1 

We took freshman data not because those only were avail- 
able, but because freshmen appeared to us to be the proper 
judges. High school seniors are a little too near some books to 

1 The data were obtained through the English Department 
of the State University of Iowa of which Professor H. C. Craig 
is the head. For this courtesy we desire to express our appre- 
ciation. 
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hare a good perspective. College seniors' ratings would have 
been the result of original preference plus four years of college 
training which would certainly tend to change their attitude 
toward some books. Seniors would tend to rate not on what 
they did like but on what they now think they ought to like. 
There would be a repetition of the "old oaken bucket" delu- 
sion. The necessity of carrying water for mother, the weight 
of well sweep, the leaks in the bucket, the spilled water, the ice 
on the well, all tend to be forgotten as distance softens the hard 
spots and enchants the pleasantness of childhood. I suspect 
that many honest grown-ups really think that in their youth 
they said at story time "Mother, tell us a story and be sure to 
put in a wholesome moral" 
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QCXBTIONNAIKB 

1. Check in the first column those books which you studied in 
class during your high school course. 

2. Check in the second column those books which you read out 
of class as collateral reading. 

3. Check in the third column those books which you now believe 
to have been of the most interest. 

4. Check in the fourth column those books which you now 
believe to have been of no particular interest, and yet not 
entirely lacking of interest. 

5. Check in the fifth column those books which you now believe 
to have been of no interest whatever, and worthless. 

6. Check in the sixth column those books which you would drop 
from a course. 

7. At the bottom of the last page you will find space. List here 
a number of books you would add to a course. 

High school last attended 

Major study in S. U. I 

Age 



Burke's Conciliation of the Colonies. 
MoCaulay's Essay on Ben Johnson. 
Macbeth. - 
King Lear. 

Midsummer-Night's Dream. 
Twelfth Night. 



The Ancient Mariner. 

Silas Marner. 

Ivanhoe. - 

Vicar of Wakefield.. 

Snowbound. 

Roger De Coverly Papers. 

Irving's Sketch Book. 

Great Stone Face. 

The Last of the Mohicans. - 

The Vision of Sir Launfal. 

The Courtship of Miles Blandish. 
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This list contains 108 books. 

Returns were received from 732 students. These 
data, were tabulated in ways to permit of convenient 
study. 

The correlations which are reported in this study 
were computed by the Pearson formula 



There are, of course, certain objections which can 
be raised about the trustworthiness of the data. I 
shall consider first the trustworthiness of the data as 
to the quantity, then as to the quality of it. 

Quantity of the Data 

The ratings of the books in this study can be 
accepted with confidence as far as the amount of evid- 
ence is concerned. We shall discuss at some length 
the reliability of the ratings because they are expressed 
in percentages. The reliability of percentages has so 
far been too little considered in scientific studies in 
education. 732 questionnaires were returned. Not 
all the books were read by every one of the 732 
students. In fact, no book was reported on by all. 
Obviously the more cases there are in any study the 
greater the reliability of the findings, other things 

1 See Thorndike Mental and Social Measurements, page 173. 
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being at all equal. We could have wished to base our 
interest ratings on 7320 cases instead of 732. Our 
evidence is much stronger because it is based on 732 
reports than it would have been if only 73 students 
had rated the books. 

Since the reliability of the ratings is a function 
of the number of reports studied we want to know 
just how reliable our findings are. To illustrate: 
Hamlet was judged by 375 students. 63% judged 
the book to be of genuine interest to them. Can we 
generalize from this and say 63 % of high school pupils 
taken at large find Hamlet of distinct appeal to them? 
Of course the ideal way to determine what percentage 
of high school pupils find Hamlet of especial fasci- 
nation would be to ask all of those who had read 
Hamlet. 

Less ideal than this way but far better than guessing 
is the determination of the reliability of the 63%. 
This can be done by the application of the formula of 
simple sampling. Each one of the 375 students, who 
having read Hamlet, rated it, can be thought of for 
our purposes as performing a little experiment. He 
is to rate Hamlet as interesting or not interesting. 
375 rate Hamlet and 63% of tbem rate it as 
interesting. Suppose we went on and got 700 ratings. 
Would 63% of the 700 rate as did the first 375? The 
percentage of 63 would probably vary slightly, But 
variations would probably center around 63. After 
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700 or 1,700 more students had been asked we should 
expect their percentage of interest to be nearer 63 
than 12 or 15 or even 50 or 90. That is, on our 375 
cases we should expect that the interest ratings of a 
much larger group would not be exactly 63 but that 
63 would be the best single bet we could make in the 
light of our facts. 

The formula -\/npq when n = number of cases, p = % 
of pupils who rate Hamlet as interesting, and q the % 
who rated the book as not interesting, will tell ub the 
standard measure of variation from 63 which the 
percentage of interest will take as more cases are 
added. 

To illustrate the process in detail: 

n = 375 students rated Hamlet. 

■p = 63% of these rated Hamlet as interesting. 

q — 37% of these did not rate Hamlet as interesting. 

With 375 cases to predict from how will the per- 
centage of interest probably vary as many more cases 
are added? 



This is found by applying formula - 
In our illustration — ™S = 2.55%. 



1 The quantity y/npq is itself divided by n to change it from 
a quantity in terms of n to a percentage. 
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This means that as more cases are added 63% may 
go up 2.66% or down 2.55%. The probability that 
the % of interest will not go over 65.55% (63 + 2.55) 
or below 61.45% (63 - 2.55) are 68 chances out of 
100. The probability of the interest in Hamlet being 
less than 57 or more than 69 is but two chances out of 
100. It is evident that, when only those cases where 
n is 80 or more are involved, the reliability of our 
findings is very high indeed. So much for the 
validity of our data as to quantity. 1 

It is possible that we have enough data but its 
quality is such that it is untrustworthy. Here the 
argument would run as follows. If you ask any num- 
ber of people a question about which they know 
nothing the answers are no better than guesses. The 
average of many fake answers is not a right answer 
except by chance. 

College freshmen may not know why they liked 
and disliked certain books in high school, but it is 
reasonable to assume the plain facts that pupils do 
like and dislike books and that they know what 
interests them and what does not. If their reports 
are false they are false not because knowledge was 



1 Those who wish to further satisfy themselves on either the 
soundness of the theory of "Sampling" or on the technique 
involved will find Yule's Introduction to the Theory of Statistics, 
Chapter XIII, a satisfactory reference. 
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lacking but because they purposely lied or reported 
with hopeless carelessness. 

There was certainly no reason for deception. There 
may have been a slight tendency to eay the proper 
thing which in this case would be to report more 
favorably than their true interests warrant on books 
which are famous, or old, or "high brow" and to 
underrate their true interest in the so-called popular 
books. The fact that the percentages of positively 
disliked books are relatively small may perhaps be 
interpreted as a result of two factors: — genuine 
dislike minus a hesitation of going on record as 
disliking books that one "ought" to like. To the 
extent that our data are vitiated by this hesitation, 
it gives a case weaker not stronger than the true 
case in favor of our Preferred List and against the 
Free Beading List. 

The following relationships of the data are our 
evidence of the validity of the data. 

The correlation between interest and a neutral 
attitude toward the books was — .18. 

The correlation between neutral attitude and dislike 
was +-51. 

The correlation between dislikes and dislike so 
great that dropping the book in question was recom- 
mended was +.67. 

These three correlations go right against any notion 
that the ratings were the product of carelessness. In 
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fact, if the markings were the product of carelessness 
rather than of thought we simply could not account 
for these correlations. 

The correlation between interest and a neutral 
attitude ( — .IS) is practically no relationship at all 
other than a chance one. And it is about what one 
could reasonably expect to be the fact. The great 
interest that some have in a book can very well run 
along with a neutral, 50-50, attitude of others. 
Unless one is under the spell of the "pathetic fallacy" 
he would not expect to be able to establish any high 
correlation between the indifference of one group and 
the interests of another. 

He would however expect more alienation to exist 
between the interests of some high school pupils and 
the positive dislikes of others than between the inter- 
est of some high school pupils and the neutral attitude 
of others. There is nothing in the nature of children 
or in education which creates a tendency of likes of 
some to correlate with the dislikes of others more 
closely than with a neutral attitude. Just the oppo- 
site would be more likely. This is exactly what 
we get. 

There is a negative correlation of significant size 
between likes and dislikes ( — .377); there is none 
between likes and neutral attitudes. This inner 
consistency of the data argues potently for the validity 
of the data. If likes and dislikes had correlated posi- 
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tively instead of negatively it would east a severe 
criticism against the quality of the data. They 
correlate negatively. 

The correlation between neutral attitudes and dis- 
likes was +.510. This shows that there is some 
positive relationship between these two series of . 
ratings. What some rate as of indifferent value, 
others tend to rate as disliked. A common sense 
interpretation of this might be that in expressing 
opinions, indifference of some predicts dislike of others. 
Perhaps timid souls mark as indifferent what more 
valiant spirits rate as disliked. 1 

One other correlation further leads us to maintain 
that our data were honestly given and represent the 
facts — the correlation between dislike and great dis- 
like, i.e., enough to advise dropping from the course, 
is +.67. This can only be interpreted if we accept 
the validity of our data. Surely there ought to be a 
high positive correlation between a tendency to dislike 
and a tendency to dislike a great deal. This relation- 
ship ought to be greatest of any in the data. It is 
the greatest relationship. 

Every correlation we have fits in with a notion that 
the pupils gave a fairly true report of their attitude 

1 If one wished to make a general observation of human 
nature concerning the reports of people it would be: indifference 
is more closely related to dislike than to positive like or interest 
as we use the term. 
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about books read. If their ratings had been false or 
overcareless no such consistency of relationship would 
have been at all probable. 

The data upon which the lists are constructed have 
a good reliability for two reasons. 

(a) The quantity of the data is too great to admit 
of a distorted picture due to inadequate sampling. 

(6) The inner consistency of the relationship 
between' 

Interest and a neutral attitude (— —.184) 
Interest and dislike (= —.377) 

Neutral attitude and dislike ( = +.510) 
Dislike and great dislike {— +.670) 

gives us plenty of evidence to accept the qualitative 
superiority of the data. All the original data and 
subsequent computations used in this study are on 
file in the College of Education, State University of 
Iowa, Iowa City, Iowa. 
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CHAPTER III 

APPLYING THE CRITERION OF COMPRE- 
HENSION TO GEOGRAPHY TEXTS FOR 
ELEMENTARY SCHOOLS 

Comprehension Values of Geography Texts 

All the tables used in this study are put together 
at the end. It is believed that this will facilitate the 
study of this research. 

This report, as part of the judgment of relative 
value of geographies, deals with the comparative 
difficulty which children find in reading the various 
texts understanding^. The method used in this 
study is of general use. No fundamental change in 
it is necessary were we dealing with history or science 
or other texts. We would define the comprehension 
value of a text as the degree to which children may 
read passages of that text and answer simple questions 
on such excerpts without entering into any intricate 
consideration of the difficulties involved Jn compre- 
hending the reading matter. We can then arrive at a 
very adequate notion of the quality which we are 
comparing. With such a formulation in mind, we 
have formed random samplings of the reading matter 
of each text book into tests and have asked simple 
06 
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questions on the basis of each sampling. We will then 
judge the value of any one book by children's responses 
to questions on its subject matter. 

There were five series of books which were candi- 
dates for selection. The series were lettered and each 
first book was called "1," and each second book was 
called "2." The names of the five series are deleted 
for obvious reasons. Throughout this study they 
will be referred to as series A, B, C, D, and E. 

One could not hold that reading difficulty or ease should alone 
decide which text to use. The other criteria of selection must 
be given due weight. Books of reading difficulty too far above 
or below the reading ability of the pupils who use the books are 
undesirable. Quite often superior subject matter necessarily 
involves slightly greater reading difficulty. For instance, a 
superficial description of the " tides " may involve easier reading 
than an adequate discussion of the ocean movements. Should 
the more difficult book be used the teacher should know where 
the hard passages are and govern herself accordingly. 

In order to ascribe to the book itself the adequacy 
or inadequacy of the responses of children to ques- 
tions asked, after the reading of any portions selected 
from it, we need to be perfectly sure of three major 
conditions that may influence the behavior, aside from 
the actual difficulty of the material in the text. We 
must know: 

1. That the samplings are a fair representation 
of the book and that the questions on the passages 
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from one book are no more difficult than the questions 
on passages form any other book. 

2. That the children who answer questions on one 
book are just as capable as those who answer questions 
on any other book. 

3. That the order in which the tests are given has 
no effect on the responses of the children who take 
them. 

The data which follow are organized to present 
results which adequately account for these three 
factors, and consequently we may attribute the values, 
as indicated through the responses of the children 
directly to the texts. 

The Fairness of the Sampling 

1. If the responses of the children to the questions 
on any one book were materially affected by chance 
inclusions of difficult passages, there would be no 
relation between the scores made by a group on the 
first halves of the tests and the scores made by the 
same group on the second halves of the tests. A group 
might then get a high percentage of the questions 
forming the first half of the selections correct, and a 
low percentage on the second half of those selections 
correct. On the other hand, if on the whole the books 
which are acclaimed as easy by selections forming the 
first half of the test (which are taken from the first 
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half of the text) are also acclaimed as easy by selec- 
tions forming the second half of the test (which are 
taken from the second half of the text), then we can 
safely decide that the difficulty of the selections is a 
function of the book. Also such agreement would 
indicate that the general difficulty of the questions 
has not been greatly influenced by the mood or ability 
of the author of the test. Agreement between the 
average difficulty of the first half with the average 
difficulty of the second half of the test, shows that the 
author's personality was not a determining factor in 
this difficulty. 

It could be maintained that all of the questions 
to one book were easy, whereas all of the questions to 
another book were hard, but we do not believe that 
this is a serious possibility because of the method used 
in construction of the tests. The tests were, of course, 
constructed with strict impartiality — passages being 
taken from the same pages of each book, and crucial 
questions being asked from each passage, with no 
consideration except the logic of the passage, as the 
author understood it. 

Table 1 gives the agreement as it is. It will be noted 
that the books which are easy to read, by the verdict 
of children's responses to the first halves are also easy 
to read by such verdict of the second halves. Those 
that are hard to read when we consider the first half 
are hard to read when we consider the last half. 
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There is only one exception, and that is D-l and D-2. 
The first halves of these two books are easy to read, 
and the second halves are hard to read. It is impor- 
tant to note that these two exceptions are of the same 
series. Those books were written in that way. 

The correlation between values by the first half 
and values by the second half is +-32. The correlation 
of values by the first half with values by the total is 
+.76. The correlation of values by the second half 
with values by the total is +.83. Though correla- 
tions with cases as few as ten are unallowable, when 
the object is to show relationship between qualities 
or between tests, it is a valuable means of showing 
characteristics of actual data, no matter what the 
number of cases. It would be unjustifiable to say 
that first halves of tests such as these correlated as 
a rule with second halves about .32, but it is entirely 
justifiable to say that these actual first halves do 
actually correlate .32 with these actual second halves. 
By Brown's formula between the totals with totals 
obtained in the same way it goes up to .48. 

It will be profitable to note in this connection that 
each second book of a series is harder than its first, 
and that the volumes I place the series about as the 
volumes II do, except the E-l and E-2, and that of all 
the series, it would be most likely that the difficulty of 
the first volume of the E series had a different relation 
to the other first volumes than the second volume of 
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this series had to the other second volumes, since 
Mr. X. WTote the first volume and Mr. Y. wrote the 
second, whereas in all of the other series, volumes 
I and volumes II were both written by one or by both 



The Equality of the Groups Used in the 
Tests 

2. Children whose responses are being evaluated to 
gain comparative estimates of the values of these 
texts are all twelve-year-olds. Since they are all of the 
same age and since there are from about 350 to about 
500 in each group, the abilities of the groups must be 
about equal, provided that their verbal intelligence is 
about equal. Table 2 gives the distributions on Wylie 
Opposites test A, B, and C, given to these same 
children about one year before the Geography Read- 
ing tests were given. These children were then 
eleven years old, and Table 2 represents the distribu- 
tions of Wylie scores for eleven-year-olds of the same 
schools that housed the twelve-year-olds who took 
the geography tests. "A" represents distributions of 
Wylie scores of eleven-year-olds January 14, 1921, 
in the same schools where the twelve-year-olds took 
the Geography Reading tests, A--1 and A-2, early in 
1922. "B" represents distributions of Wylie scores 
for eleven-year-olds January 14, 1921, in the schools 
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where the twelve-year-olds took the Geography 
Reading testa, B-l and B-2, early in 1922, etc. 
Table 3 gives these same facts for twelve-year-olds, 
January 14, 1921, by groups of schools which took the 
various Geography Heading tests. Table 3 gives the 
facts which insure us that it is not characteristic of 
any of these groups of schools to have twelve-year-olds 
which are much brighter than eleven-year-olds, and 
that therefore the relative abilities of the twelve-year- 
olds in these groups of schools will be just about what 
those same children were one year before, when they 
were eleven-year-olds. 

In Table 2 the percentage of zeros was for the 
"A" group, .050; for the "B" group, .070; for 
the "C" group, .070; for the "D" group, .067; for 
the " E" group, .058. The lines punctuating the distri- 
butions are percentile lines. The first line in each 
distribution marks off the lowest 10% from the other 
90%. The next line marks off the lowest 25% from 
the highest 75%. The third line marks the lowest 
50% from the highest 50%. The fourth line marks 
the lowest 75% from the highest 25%. The last line 
marks the lowest 90% from the highest 10%. The 
remarkable correspondence of the position of these 
lines in the five groups in both Table 2 and Table 3 
shows beyond any doubt that the groups of children 
are identical, so far as ability on the Wylie test is 
concerned. 
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Negligible Effect of the Order of Giving the 
Tests 

3. The possibility that the order in which tests were 
given might affect the responses of the children and 
would therefore introduce a factor in the evaluation 
which could not be justly attributed to the book itself, 
needs consideration. It has been repeatedly found 
that tests constructed as these were, are not easily 
amenable to practice. We found at Garden City that 
giving practice in this function so as to make an 
appreciable difference in the average of the groups 
took two years of special training; and Dr. Arthur 
Gates found at Scarborough, where the abilities were 
already near the intelligence limit, that practice in the 
function made no difference at all in the test. 

All the "2's" were given directly after the "l's," 
and still each "2" is harder than it« corresponding 
" 1." This seems to indicate that the order of giving 
has no great influence on the results. A-l, B-l and 
C-l were given before any other test, and 0-1 and E-l 
were given after one of the other three, both volumes 
of the series, had already been administered. In spite 
of this fact, B-l and C-l are pronounced the two 
easiest, and A-l, the hardest to read. This would 
seem to indicate that the practice gained from previous 
experience in the test had no great influence on the 
results. 
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Some schools were chosen in which to change the 
order of the giving of the tests. In considering these 
results we must bear in mind that the determination 
for a single school 1 is very unreliable, and if the 
general tendency of the results is like the results of the 
total, in spite of the change in order of administration, 
our conclusions will be verified. In No. 1 School 
the order of giving was E-l, D-l, C-l, B-l, — with the 
following results: 

Pes Cent or Correct 
Text Responses 

B-l .76 

E-l .67 

C-l .66 

D-l .63 

There were seventeen 12-year-olds. 

In School No. 2 the order of giving was E-l, E-2, 
D-2, C-2,— with the following results: 

Pes Cent or Correct 
Text Responses 

E-2 .75 

E-l .70 

D-2 .69 

C-2 .57 

There were one hundred and fourteen 12-year-olds. 

1 The data used here were gathered in a middle western 
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In School No. 3 the order of giving was D-l, E-l, 
C-2, A-2, with the following results: 





Per Cunt of Corbxct 


Text 


Responses 


E-l 


.81 


D-l 


.75 


A-2 


.67 


C-2 


.66 



There were forty-six 12-year-olds. 

In School No, 4, the order of giving was D-l, D-2, 
E-l, C-2, — with the following results: 





Per Cent of Correct 


Text 


Responses 


E-l 


.67 


D-l 


.65 


D-2 


.57 


C-2 


.54 



There were sixty-eight 12-year-olds. 

In School No. 5, the order of giving was E-l, E-2, 
C-2, D-2, with the following results: 





Feb Cent of Correct 


Text 


Responses 


E-l 


.59 


D-2 


.58 


E-2 


.57 


C-2 


.51 



There were fifteen 12-year-olds. 
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In School No. 6, the order of giving was D-2, E-l, 
E-2, C-2,with the following results: 





Feb Gent of Correct 


Test 


Responses 


E-2 


.39 


E-l 


.37 


C-2 


.27 


D-2 


.22 



There were nine 12-year-olds. 

Certain noticeable features deny "the order of 
giving" any part in the results. Certainly the test 
which is given last is not always the easiest. The only 
case in which it is easiest is in School No. 1, and there 
it is because B-l was given last, and B-l is by our total 
result the easiest. Also there is no case in which the 
test which is given first is demonstrated to be hardest. 
E-l is always about as hard as D-l, as it is in our totals, 
no matter what the order of giving. As a matter of 
fact, E-l was easier for the children of School No. 3 
School No. 4 and School No. 5 than D-l, even though 
E-l was given first in No. 4 and No. 5, and D-l was 
given first in School No. 3. A-2 bears the position in 
difficulty in School No. 3 that it has in our totals, even 
though it was given last. C-2 is the hardest for the 
children at No. 1, No. 3, No. 4 and No. 5, and next to 
the hardest for the children of No. 6, even though it 
was given last, or next to the last in each, — that is to 
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say, C-2 which is hardest by our summaries when 
practice in only one test (CM) had been given, is 
also hardest in these special schools when practice in 
two or three testa had been given before the adminis- 
traction of C-2. E-2 is easier than C-2 and D-2 in No. 
2 and No. 6 even though it is given before C-2 
and D-2, and is about as hard as D-2 and easier than 
C-2 in School No. 5, even though it is given before C-2 
and D-2. 

This analysis of the schools where a special Order of 
giving was instituted seems to indicate that the data 
are independent of the order in which the tests were 
given. 

How Reading Difficulty of the Texts Was 
Obtained 

The reports came to us from each school, listing 
the number of correct responses, the number of 
incorrect responses, and the number not attempted, 
for each question. We were therefore able to deter- 
mine the percentage that the correct responses were 
of the total attempts. As indicated above, we threw 
out of consideration any school in which the "not 
attempted" was larger for any questions at the end 
than for questions at the beginning of the test, since 
this indicated that insufficient time had been given 
to adequately measure the responses of the children 
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to the material itself. Thia "percentage correct" 
for each book constitutes the third column in Table 1. 
We have here a picture of the relative ease with which 
twelve-year-olds read the ten volumes in question. 
Table 4 gives these percentages by series. 

As it is somewhat difficult to interpret the meaning 
of these differences, we have translated them into a 
more readiiy understood medium. Table 5 shows 
the deviations in terms of Standard Deviation and 
the average age necessary to read the texts. The 

percentage of correctness was translated into zrfr. 

deviation in terms of the variability of twelve-year- 
olds. This deviation was translated into a scale 
ranging from five standard deviations below the mean 
to five standard deviations above the mean, — calling 
the former zero, and the latter 100. This places the 
average at 50. We have values now for each book 
which correspond to McCaU's T-score on the Thorn- 
dike-McCall Reading Scale. In Table 5, B-l is as 
hard as 43 on the Thorndike-McCall Reading Scale. 
C-l is as hard as 45 on the Thorndike-McCall Reading 
Scale, etc. 

Whereas these terms are indispensable statistically 
to arrive at a final simplified report, they are in them- 
selves so intricate that it is very important to present 
our relative values in a manner more easily under- 
stood. Once arrived at a T-score, it is a simple 
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matter to use the Thorndike-McCall age norms to 
find what age is necessary to comprehend any one 
book. B-l is as hard as 43 on the reading scale, and 
the average age of the T-acore of 43 is, according to the 
McCall norms, 130 months. We can therefore, say 
that children must be this old in order to understand 
B-l. So with the other reading ages in Table 5. 
It is also of value to us to know which is the lowest 
grade able to read each book. These values are 
derived through the grade norms published by McCall 
for T-score of his test. We now have before us in 
Table 5 the average age necessary to read each book 
and the average grade necessary to read each book. 

Selecting Texts to Meet the Needs of Particu- 
lar Grades and Students 

One other immediate advantage, other than the 
selection of texts, should be parenthetically answered 
here. By test the mental age of grades used in this 
study was determined. Thus we could see which 
grades in any one school are like ft fourth grade normally 
would be; which grades are like a fifth grade normally 
would-be, etc. In this way individual needs of each 
grade of this city's schools can be seen at a glance. 
As these scores are on a"verbal intelligence test, the 
differences are in exactly the same ability upon which 
the relations between texts are based. The correla- 



ted t, Google 



80 TEXTBOOK SELECTION 

tion between the Thoradike-McCall and the Wylie 
is high. 

A-l might very well be the beet book in every other 
respect than comprehension. In fact, its very values 
may be associated with a necessary sacrifice in terms 
of comprehension. In that case it might be very- 
advisable to use A-l in all schools where they are 
able to comprehend it, and to use the book which 
may not be as valuable, but easier to read, only in 
those schools where it is necessary to sacrifice the 
other values in favor of the sine qua non, comprehen- 
sion. The statement that A-l is better in other ways 
is of course entirely hypothetical. 

The Variability of Reading Difficulty Within 
a Single Text 
There is a further characteristic of geography text 
books which our data will allow us to analyze. The 
difficulty of a book as a book is one major considera- 
tion in the treatment of comprehension value, but 
the degree to which the difficulty varies from very 
easy to very hard is a very important supplementary 
consideration. B-l is easier than any of the other 
volumes- 1 to read. However, if this average ease of 
comprehension were the result of the product moment 
of very easy and very hard passages, it would never- 
theless be an undesirable text. We want a book the 
difficulty of which is reasonably constant throughout. 
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Tables 6 and 7 present the T-scoree and reading 
ages of each question. The reading age is to be 
interpreted as the average age necessary to read the 
passage and answer the questions as per the conditions 
of our test. Some of the disparity in ages is of course 
due to the sampling of passages and the chance diffi- 
culty of the questions, but the range of difficulty of 
the text book is certainly pictured in a rough manner 
by the range in difficulties of these passages. 

It is especially of note that the difficulty of the 
sentences in D-l and D-2 marches progressively from 
easy to hard, which fact is already noted in connection 
with Table 1, where the first halves of these two books 
were shown to be easier than the second halves. The 
questions are in serial order, and samplings are from 
the beginning to the end of the book. Whether or not 
this feature is desirable is dependent upon the progress 
which children can make in ability to read. The 
difference in difficulty between the end and beginning 
of the D's is not justified by the progress which 
children make in the time covered by the books. 

Table 8 gives the standard deviation of reading 
ages of the questions of each book and the average 
reading age of the book. It is because of dropping of 
decimals that the average age of questions per book 
does not check with the age necessary to read the 
book, — computed directly. However, the order of the 
books is of course the same. A consideration of these 



aii^cd by Google 



82 TEXTBOOK SELECTION 

measures of variability of the difficulty of | 
from the various texts expressed in terms of age 
shows that as a tendency, the first and second volumes 
of a series have variabilities very much alike. Note 
that B-2 and B-l are together; note that D-l and D-2 
are together; note that A-l and A-2 are very close 
together ; note that E-l and E-2 are fairly close together. 
C-2 and CM are very different in variability, but all the 
other series have Standard Deviations of the volumes-1 
which show a likeness to the Standard Deviations of 
volumes-2. The highest Standard Deviations are for 
A-2, C-2 and A-l. These three volumes are among 
the four most difficult to read (Table 5). The lowest 
standard deviations are those for C- 1 , E-2, B-2 and B-l . 
All of these, except E-2 are among the five books which 
are easiest to read (Table 5). 

There is then, luckily, an association of comprehen- 
sion virtues in these texts. Those that are hard to 
read are also spotty in their difficulty and those that 
are easy to read have an even tenor of difficulty. 

The Success op the Texts in Reaching Theib 
Reading Public 

Table 9 gives the frequency of each reading age 
of all questions on all volumes-1. An analysis of 
this frequency distribution of average age necessary 
to answer each of 104 questions, after reading the 
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passages selected from five geography text books 
meant for beginners in geography in grades 4 and 5, 
will allow us to judge how far the authors of the texts 
have reached their public. Their public is, of course, 
the children themselves, — not what teachers think of 
those children. Although the sales today may be 
largely based on what superintendents and teachers 
think children of the fourth and fifth grade can read, 
rather than what they actually do read, it is just such 
data as these which will change that emphasis and 
allow us to judge text books upon a basis of their real 
ability to get across to the children that they are 
meant for. The average of the average age necessary 
to read these questions is 135.79 months. With a 
standard deviation of about a year and a quarter, 
about five-sixths of the material is too hard for children 
in grades 4 to understand. Since one standard 
deviation below the mean of the distribution, repre- 
sented in Table 9, is at about 120 months, if we count 
the children in grades 4 as nearly all younger than 120 
months in reading ability, five-sixths of the passages 
of the texts given them are too hard for the group. 
Volumes-2 are a little more adequate to the task 
which they pretend to perform. The average of the 
average age necessary to read the 111 questions on 
volumes-2 is 143.50 months. The standard devia- 
tion of reading age of questions is again about a year 
and a quarter. This distribution of reading ages by 
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questions on volumes-2 will be found in Table 9. If 
vohimes-2 are meant primarily for grades 6 and 7, 
only about one-half can be understood by the children 
of the grade in which they are first introduced, (Grade 
6.) Since the average age necessary is about twelve 
years/ if we count most sixth-graders as not more than 
twelve years old in reading ability they are then able 
to understand only one-half of the matter presented 
to them in all geography texts meant for the grade. 
This one-half is not the first halves of the book as is 
apparent from Table 7. 

Individual Differences in Needs of Various 
Grades 

On January 14, 1921, the 5-A grades of the schools 
used in this research, about to enter on their 6th 
grade work had average scores on the Wylie A, B and 
C ranging from 55.15 to 19.69. Interpreted, this 
means they range from an average age of 12 years 
6 months to S years 6 months, in the ability measured 
by the Wylie test. They differ in the ability of work 
which they can do as measured by the Wylie test 
from average 7th grade capacity at beginning of B 
work to average 4th grade capacity (at point of 
promotion from B to A). 

Associating these facts with those of Table 5, it is 
apparent that some 6th grades are able to use any of 
the texts, whereas others need to choose the very 



aii^cd by Google 



TEXTBOOK SELECTION 85 

easiest and then still will have a book which is too 
difficult. The beat 6th grades can just read the hardest 
volume-2 and not all can read the easiest. This is a 
very valuable practical consideration since we may be 
able to determine as suggested above that a certain 
book excels the others in all qualities save ease of 
reading. Then we must know in which grades the 
difficulty of comprehension makes no difference in 
order to benefit where we can by the excellence of the 
book in other qualities. 

In order to find which 6th grades (and for volumes-1 
which 4th grades) are able to use & difficult book, 
consult the mental age ratings of each grade. 

The Text Already in Use Had Negligible 
Advantage 
In conclusion a word regarding the geography now 
in use in the lower grades is necessary to assure us that 
training in it has not prejudiced our results in favor 
of one of the books. "B," First Book, Part One has 
been used the last three years. Since the first 90 
pages of the 256 (exclusive of indices) of our B-l is 
this same Part One, we must be sure that questions 
on this portion are at least as hard as questions on the 
remainder of B-l. This portion that has been studied 
covers questions on our test 1 to 5 inclusive. Though 
three of these five questions were comparatively easy 
for our 12-year olds (see Table 6) still Table 1 shows 
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that the second half of B-l (which was not studied) 
was easier than the first half (part of which was 
studied). We feel therefore that no important advan- 
tage was given the "B" series. 

This treatise is of comprehension value of the texts 
only. It makes no claims to ascertain any of the 
other necessary virtues of a text. When those too are 
credited through objective means we will be able to 
choose texts, not risk them. 

Table 1 





Percentage of 


Percentage of 


Percentage of 




questions on 


questions on 


questions on 


Key number 
of text 


first half of 


second half of 


total of test 


teat answered 


test answered 


answered cor- 


correctly by a 


correctly by a 


rectly by a 




normal group 


normal group 


normal group 




of 12-year-olds 


of 12-yew-olds 


of 12-year-olds 


B-rx 
04' 


68 


77 


72 


71 


67 


69 


D-I 


79 


64 


66 


E-l / 
B-2 7 


64 


68 


66 


63 


64 


64 


E-2 


62 


66 


64 


A-l 


62 


61 


62 


D-2 


64 


61 


68 


A-2 


62 


69 


66 


C-2 


54 


48 


61 
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Table 2. — Distributions in Wylie A, B and C, 

January 14, 1921 bt Groups of Schools 

Taking A, B, C, D and E Geography 

Reading Tbsts-Elbtbn-teab-oldb in 

Tbobe Schools at That Time 





A 


B 


C 


■> 


X 







IB 


34 


31 


41 


33 




0- 6 




20 


24 






-HO % of the number Mow thi» 


5- 10 




21 


18 


36 


35 




10- 10 





fJ 




10 


12 




16- 20 




7 


14 


18 






30- 25 






17 


IS 


22 


->35 % of the number below tbii 


25- 30 


12 


34 


17 


29 


27 


line in each attribution. 


80- 35 


20 


15 


26 








35- 40 


£0 


18 


21 


» 


31 








23 


25 


24 


32 




45- SO 


20 


36 


27 


36 




— tThia Una halves the total 


00- 65 


26 


47 


37 


46 


56 


number in each distribution. 


55- 80 


28 


3i 


32 


48 


47 




60- 66 


37 


61 


35 


M 


46 


—►25 % ol the number above thin 


66- TO 


26 


44 


29 






line in eaoh diitribu tion. 


70- 75 




23 


28 


37 


24 


•10 % of the:number above thil 


76- 80 


15 


14 


17 


17 




line In each dintribution. 


SO- 86 


13 


13 






17 




85- 90 


8 




8 


10 


7 




DO- OS 


6 


7 





12 






06-100 


4 


7 




S 


6 




100-106 


a 






fl 


a 




106-110 







2 


t 







110-116 


1 


1 


2 


* 







120-135 






1 




i 






344 


186 


448 


612 


m 
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Table 3. — Distributions in Wtlib A, B, and C, 

January 14, 1921, bt Groups of Schools 

Taking A, B, C, D and E Geography 

Reading Tests — Twelvb-yeab-olds in 

Those Schools at That Time 





* 


B 


• 


D 


X 







18 


30 


17 


28 


85 




0- s 




10 


17 


27 


17 




8- 10 




e 




23 


12 




10- IB 


4 


7 




12 






IS- 20 


a 


7 


10 


18 


14 




20- 96 


3 


12 


11 








25- 80 


7 




12 


17 


16 








32 


10 


23 


23 




35- 40 


14 




23 


25 






40- IS 


20 






84 


23 




IS- 60 


10 


JO 


24 


37 


28 




60- 66 


37 


30 




61 


58 




56- 60 


87 


28 


34 


40 


40 




00- 06 


35 


30 


37 


58 


60 




66- 70 


30 


40 


35 


50 


46 




70- 7S 


21 


23 


33 


33 


84 




75- SO 




31 


28 


31 


36 




SO- 86 


20 


IS 


25 


32 


23 




85- 00 


IB 






15 


10 




to- es 


9 


6 


13 


13 






SS-100 


S 


2 




a 


8 




100-105 


2 




8 


6 


B 




106-110 











S 




110-115 


I 





3 


\ 


4 





120-136 




2 




3 






126-130 











2 




130- 185 







1 











303 


3.7 


444 


613 


566 
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Table 4 






PERCEMTAOE OF QUESTIONS 


Text Series 




COBBECT 


E 




67.7 • 


• E 




M.8- 


D 




62.1 


C 




59.4 


A 




58.6 
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Table 5. — Distinctions in Compbehension Value 
Translated into Age and Grade 



T«tt 


Proportion 


normal dis- 
tribution of 
difficulty 


like T'horn- 
dlke-Mo- 

Call raid- 
ing .cole 


prehend 


prehend 






B-l 
C-l 

E-l 
B-2 
B-l 

A-l 
D-2 
A-2 
C-2 




72 
69 
86 
68 
64 
64 
82 
58 
88 
81 


; 


58 
SO 

SS 
38 
BI 
20 
45 
02 


45 
45 

48 
46 
43 
48 
49 


a 



4 
9 

5 

8 


133 
138 
138 
138 

180 
141 
141 
145 
ISO 


8 

I 

6 
5 

G 
5 
8 
• 

8 


48 
50 
88 
88 
74 
74 
82 
00 
09 












•raien 








Note columns above lor 
Me and grids par volume 


B 

D 
C 




88 

95 
82 
69 
69 


; 


47 

30 
31 
23 
23 


45 

48 
48 
47 
47 


s 

9 
7 


135 
138 
141 

14S 


a 
B 


51 

so 

82 
95 
05 
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TABLE 6. — T-BCOREB AND READING AGES BT QUES- 
TIONS in Tests on Volumes-1 



Number of 
j question 


I 


|s 


! 


h 


I 


h 


J 


3 

Is 


I 


I> 




B-l 


C-l 


D-l 


B-l 


A-l 


s 

r» 

s 
a 

T 
8 

10 
11 
13 
13 
14 
IS 

IT 
IB 
19 
20 
91 
99 
33 
31 


40 
35 
IS 

U 

45 
49 

46 

se 

38 
IS 

38 
39 
43 

42 


3 
3 


S 

7 

e 

3 

3 
3 

5 

e 


12 
10) 
13 
14 

13 

141 
13) 
16 
IS) 
11 
13 
14) 
11 
11 
131 
13 
13! 
131 




48.2 
48.7 
38.2 
43.3 
43,9 
43. S 
51.0 
42.8 
40. G 
40. 8 
45. 3 
42.9 
45.3 
44.2 
51. 
43.3 
*2.S 
48.5 
47.7 


145 
140 

117 
131 
130 
132 
152 

122 

136 
130 

136 
IB4 
152 

128 
148 
143 


M 
39 
33 
39 

43 

38 
48 
43 

42 

u 

48 

II 

38 
48 
47 

39 
51 

56 

50 


3 
3 

8 
6 
6 
8 

7 



6 

3 

2 

« 

8 
8 

3 

8 

5 


119 
119 
102 
120 
133 
132 
113 

132 
134 
130 
138 
145 
161 
137 
113 
137 
143 

US 
153 
187 
152 
151 


34 

62 
44 
45 
37 
54 

48 

37 
49 

« 

45 
43 
42 
40 


2 
3 
5 

7 
7 
S 
7 
9 
8 
5 
t 




e 

5 
2 


108 
125 
157 
135 
139 
115 
163 

140 
138 

148 
123 

141 
138 

149 
134 


43 

40 
50 
35 
47 

48 
54 

42 
43 
46 
46 
48 
54 
47 
56 

38 
42 
41 


6 
8 
3 

5 

3 

1 
8 

8 
9 

9 
7 
7 
7 
5 
7 

6 

6 
9 


139 
123 
161 

110 
143 
134 
138 
145 
161 
183 
129 
130 
141 
140 
146 
163 
143 
169 
155 
112 
127 
126 
130 
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Table 7. — T>8COBES and Reading Ages by Ques- 
tions in Tbstb on Volumes-2 



Number of 




| 


E 


I 


E 


I 


E 


I 


! 


9 


***** 


i 


IS 


fi 


h 


J 


I * 


1 


!» 


h 




B-2 


E-3 


D-3 


A-2 


C-3 




40 


1 121 


53.0 


156 


4S.3 


136 


57.1 


100 


62.8 


157 




63 






142 


41.8 


130 


55 


5 


166 


44 


8 


135 






135 


48.0 




45.0 


138 


50 


2 


170 


40 





1*1 




47 


S 143 


44.8 


136 




138 


M 





172 


42 





127 






7 144 


43.0 


120 


51.3 


153 


58 




173 


40 


4 


130 




47 


7 144 


61.0 




53.0 


161 


50 




170 


56 


S 


166 




SO 


8 1S3 


41.6 


I2fl 


40.3 


130 


40 




138 


54 


7 


163 




45 


3 136 


41.3 


125 


37.7 


116 


40 


2 


148 


56 




107 




40 


1 138 


53.8 




40.8 


123 


41 


3 


125 


37 


2 


11* 




43 


3 128 


43.0 


127 


43. a 




40 




147 


40 


7 


140 




14 


2 134 


40.5 


122 


47.7 


143 


42 





127 


52 


S 


150 






7 149 


51. 8 


164 


51.8 


164 


42 





127 


57 


7 


171 




93 


165 


52. 3 




47.2 


142 


35 











100 




IS 


g 138 


44. 6 


135 


03.1 


168 







126 


50 


8 


102 






4 163 




133 


63.0 


100 




2 


125 


47 


2 






38 


3 117 


43.0 


133 


47.5 


113 






100 


36 


B 


112 






7 US 


45.6 


137 


61.0 


162 




5 


148 


56 


8 


166 






3 136 




143 


50.5 


177 







153 


51 


5 


164 




u 


7 1*0 






43.6 


132 


« 




107 


38 


7 


118 


3D 




6 133 






46.0 


188 


48 




140 


50 





100 


31 


4G 


9 138 






43.0 


130 


45 




186 


53 





101 


n 










43.0 


132 


47 


7 


143 


53 


3 


150 


33 










51.6 


163 


42 





120 






34 










60.4 


178 


43 










as 














46 


3 


130 






M 














40 


5 


123 
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Tablb 8. — Standard Deviations and Averages of 

Reading Ages of the Questions by Book 

in Order of Size of Standard Deviations 



Book 


S.D. 


M 


C-l 


9.45 


134.58 


E-2 


11.48 


139.11 


B-2 


11.79 


138.86 


B-l 


14.55 


130.95 


E-l 


14.64 


136.37 


D-2 


15.56 


143.79 


D-l 


16.79 


135.75 


A-l 


17.32 


140.35 


0-2 


17.66 


148.14 


A-2 


18.79 


146.08 
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94 TEXTBOOK SELECTION 

Table 9. — Distribution of Reading Aqe Neces- 
sary to Read Questions on Volumes- 1 and 
Volumes-2 



Reading age 


Volumes-1 


Volumes-2 


100-105 


1 




105-110 


2 




110-115 


5 


3 


115-120 





4 


120-125 


7 


4 


125-130 


11 


13 


130-135 


15 


8 


135-140 


15 


19 


140-145 


12 


15 


145-150 


9 


5 


150-155 


8 


11 


155-160 


1 


9 


160-165 


4 


5 


165-170 


4 


8 


170-175 




3 


175-180 




4 


180-185 


1 






Volumes-1 


Volumes-2 


N 


104 


111 


M 


136.20 


143.95 


S.D. 


15.25 


16.05 
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RETURN TO the circulation desk of any 
University of California Library 
or to the 
NORTHERN REGIONAL LIBRARY FACILITY 
Bldg. 400, Richmond Field Station 
University of California 
Richmond, CA 94804-4698 

ALL BOOKS MAY BE RECALLED AFTER 7 DAYS 
2-month loans may be renewed by calling 

(415) 642-6753 
1-year loans may be recharged by bringing books 

to NRLF 
Renewals and recharges may be made 4 days 

prior to due date 

DUE AS STAMPED BELOW 

'ftM 27 1991 



YB 0525! 
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